A simplified pneumonia severity index (PSI) for clinical outcome prediction in COVID-19

Background The Pneumonia Score Index (PSI) was developed to estimate the risk of dying within 30 days of presentation for community-acquired pneumonia patients and is a strong predictor of 30-day mortality after COVID-19. However, three of its required 20 variables (skilled nursing home, altered mental status and pleural effusion) are not discreetly available in the electronic medical record (EMR), resulting in manual chart review for these 3 factors. The goal of this study is to compare a simplified 17-factor version (PSI-17) to the original (denoted PSI-20) in terms of prediction of 30-day mortality in COVID-19. Methods In this retrospective cohort study, the hospitalized patients with confirmed SARS-CoV-2 infection between 2/28/20–5/28/20 were identified to compare the predictive performance between PSI-17 and PSI-20. Correlation was assessed between PSI-17 and PSI-20, and logistic regressions were performed for 30-day mortality. The predictive abilities were compared by discrimination, calibration, and overall performance. Results Based on 1,138 COVID-19 patients, the correlation between PSI-17 and PSI-20 was 0.95. Univariate logistic regression showed that PSI-17 had performance similar to PSI-20, based on AUC, ICI and Brier Score. After adjusting for confounding variables by multivariable logistic regression, PSI-17 and PSI-20 had AUCs (95% CI) of 0.85 (0.83–0.88) and 0.86 (0.84–0.89), respectively, indicating no significant difference in AUC at significance level of 0.05. Conclusion PSI-17 and PSI-20 are equally effective predictors of 30-day mortality in terms of several performance metrics. PSI-17 can be obtained without the manual chart review, which allows for automated risk calculations within an EMR. PSI-17 can be easily obtained and may be a comparable alternative to PSI-20.

The conclusions drawn in this study elegantly stem from the meticulously designed methods, presenting a logical approach for studying pneumonia severity in the context of COVID-19.By retrospectively analyzing a sizable cohort of hospitalized COVID-19 patients, the researchers effectively compared the performance of the simplified Pneumonia Severity Index (PSI-17) with the original PSI-20.Through rigorous statistical analyses, including correlation assessments and logistic regressions for 30-day mortality prediction, the study robustly evaluated the predictive abilities of both indices.The findings, indicating comparable performance between PSI-17 and PSI-20 across multiple metrics, underscore the feasibility and reliability of the simplified index.Moreover, the pragmatic adaptation of PSI-17, eliminating the need for manual chart review, underscores the study's commitment to enhancing clinical utility and efficiency.Thus, the study's conclusions not only reflect the logical progression of its methods but also provide actionable insights into optimizing pneumonia severity assessment in COVID-19 patients.
The authors thank the reviewer for the positive and insightful comments.
We have updated the discussion with these references, especially to note that the PSI is not comprehensive on known risk factors, as the reviewer points out with the suggested references.For instance: "The PSI is not comprehensive and other model features could be added, such as diabetes, which is a known risk factor for poor outcomes in COVID-19 [ref] and is not one of the listed co-morbidities in PSI (Table S1)." We have also included the excellent review of community acquired pneumonia in response to point 2 of reviewer #2 (see below).The original Fine et al study was already referenced in the manuscript.

Reviewer #2:
Manuscript ID: PONE-D-24-00824 A Simplified Pneumonia Severity Index (PSI) for Clinical Outcome Prediction in  Summary: This study shows that a simpler version of the PSI score has equivalent forecasting ability for COVID-19 than the original version.

Major findings:
The PSI score has 20 variables of which 17 can be obtained from EMR and the last 3 must be evaluated manually from the patient's charts.A comparison of a 17 variable PSI score with the original 20 variable version showed little loss of prediction ability of COVID mortality.

Strengths:
This is a large database of 1138 patients from the beginning of the COVID-19 pandemic.
The authors thank the reviewer for the positive review and concise summary.

Major comments / Weaknesses:
1.This is a special application of the PSI 17-score to a specific disease (COVID-19) in a specific period.The mortality found in the onset of the pandemic is not the same as the mortality found now.Could the authors comment on current the COVID-19 situation.
We agree that the mortality found throughout the pandemic waves differs.While this study assesses the PSI-17 as a mortality prediction tool in the early pandemic era, we see this as a proof of concept for the comparability of PSI-20 and PSI-17.To conclusively determine if PSI-17 predicts mortality during other variant eras, the PSI-17 would need to be studied in other variant eras to determine predictive ability in these other eras.This analysis is beyond the scope of this article, but we plan to use PSI-17 to study subsequent variant eras.We have updated the Discussion for improved clarity on this point: "Similarly, as the mortality has decreased over time through evolving SARS-CoV-2 variants [refs], the application of a mortality prediction tool would need to be validated in different variant eras." 2. Could this simpler PSI version be extrapolated to community acquired pneumonia or is this a COVID-19 special case related to pleural effusion, altered mental status and skilled nursing facility?
The authors thank the reviewer for the insightful comments and thought-provoking questions.While we hesitate to speculate about the ability of PSI-17 to accurately predict mortality in community acquired pneumonia (CAP), we do think the simplified PSI-17 tool would be easier to implement in clinical care or research on CAP.Further study is needed to validate PSI-17 in CAP.We have added to the Discussion: "To determine whether PSI-17 is predictive of mortality in community acquired pneumonia [ref], validation is an appropriate cohort would need to be performed."

Minor comments:
1. How long did it take the researchers to assemble the 3-variable data (pleural efussion, altered mental status and skilled nursing facility) from the 1138 patients?
The authors thank the reviewer for the insights and great questions.Data extraction for the 3 variables from PSI-20 which required manual extraction into a REDCap database was an intensive process, in part because of the subjective nature of these variables.Supervision for 10 reviewers (senior research coordinators and medical students) was performed by 3 senior investigators who conducted training, answered queries from reviewers, resolved discrepancies and reviewed in duplicate 20% of the cases to validate individual reviewers.In total, we estimate approximately 400+ person-hours were spent in manually extracting these variables.